Accuracy assessment of High Resolution Satellite Imagery by Leave-one-out method

Interest in high-resolution satellite imagery (HRSI) is spreading in several application fields, at both scientific and commercial levels. A fundamental and critical goal for the geometric use of this kind of imagery is their orientation and orthorectification, the process able to correct the geometric deformations they undergo during acquisition. One of the main objectives of the studies about orthorectification is the definition of an effective methodology to assess the spatial accuracy achievable from orthorectified imagery. Currently, the most used method (hold-out validation HOV) to compute this accuracy just consists in partitioning the known ground points in two sets, the first used into the orientation-orthorectification model (GCPs – Ground Control Points) and the second to validate the model itself (CPs – Check Points); in this respect, the accuracy is just the RMSE of residuals between imagery derived coordinates with respect to CPs coordinates. However this method has some drawbacks: it is generally not reliable and it is not applicable when a low number of ground points is available. First of all, once the two sets are selected, accuracy estimate is not reliable since it is strictly dependent on the points used as CPs; if outliers or poor quality points are included in the CPs set, accuracy estimate is biased. In addition, when a low number of ground points is available, almost all of them are used as GCPs and very few CPs remain, so that RMSE may be computed on a poor (not significant) sample. In these cases accuracy assessment with the usual procedure is essentially lost. In the present work we propose an alternative to the previously described method to perform a spatial accuracy assessment, that is the use of the Leave-one-out crossvalidation (LOOCV) method for the orientation and orthorectification of HRSI. The method consists in the iterative application of the orthorectification model using all the known ground points (or a subset of them) as GCPs except one, different in each iteration, used as CP. In every iteration the residual between imagery derived coordinates with respect to CP coordinates (prediction error of the model on CP coordinates) is calculated; the overall spatial accuracy achievable from the orthorectified image may be estimated by calculating the usual RMSE or, better, a robust accuracy index like the mAD (median Absolute Deviation) of the prediction errors on all the iterations. In this way we solve both mentioned drawbacks of the classical procedure: it is a reliable and robust method, not dependent on a particular set of CPs and on outliers, and it allows us to use each known ground point both as a GCP and as a CP, capitalising all the available ground information. To test this method we modified the software SISAR, developed by the Geodesy and Geomatics Team at the University of Rome “La Sapienza” to perform rigorous orientation of HRSI, integrating it with a module suited to carry out iteratively the core algorithm with point configurations required to apply the Leave-one-out method. The software was tested on EROS-A1 and Quickbird imagery, 7th International Symposium on Spatial Accuracy Assessment in Natural Resources and Environmental Sciences. Edited by M. Caetano and M. Painho. 534 confirming the good features of the Leave-one-out method. Moreover SISAR was compared to the world recognized commercial software OrthoEngine v. 10 (PCI Geomatica), which required manual iterations to realize the Leave-one-out procedure; this comparison showed the quite good performances of the SISAR rigorous model.